Skip to content

Codex/assertion text diff blocks#14425

Merged
nicoddemus merged 22 commits into
pytest-dev:mainfrom
hamza-mobeen:codex/assertion-text-diff-blocks
May 18, 2026
Merged

Codex/assertion text diff blocks#14425
nicoddemus merged 22 commits into
pytest-dev:mainfrom
hamza-mobeen:codex/assertion-text-diff-blocks

Conversation

@hamza-mobeen
Copy link
Copy Markdown
Contributor

@hamza-mobeen hamza-mobeen commented Apr 27, 2026

closes #6757

Description

This PR introduces a new configuration option, assertion_text_diff_style, to allow users to customize how multiline string assertion failures are displayed.

As highlighted in #6757, when pytest uses ndiff to display differences for multiline strings (like output from capsys), it can sometimes become an unreadable "soup", especially when changes involve indentation or leading/trailing whitespace. The current behavior often results in a confusing line-by-line diff that is difficult to parse.

This PR adds an optional block diff style that displays the exact contents of the "Left" and "Right" text blocks intact and sequentially, making it significantly easier to read the literal multiline strings being compared without the interspersed ndiff noise.

Key changes:

  • Added assertion_text_diff_style configuration option. Defaults to ndiff to preserve backwards compatibility.
  • Implemented the block diff style (_diff_text_block) for multiline string comparisons, solving the readability issue raised in assertion diffs for multiline-string can become unreadable soup #6757.
  • Added fallback logic so that single-line comparisons still use ndiff even if block is selected.
  • Added strict validation to ensure assertion_text_diff_style is either ndiff or block.
  • Included extensive test coverage in testing/test_assertion.py verifying block formatting, fallback logic, blank line handling, and configuration validation.

Checklist

  • Include documentation when adding new features.
  • Include new tests or update existing tests when applicable.
  • Allow maintainers to push and squash when merging my commits. Please uncheck this if you prefer to squash the commits yourself.
  • Add text like closes #XYZW to the PR description and/or commits (where XYZW is the issue number).
  • If AI agents were used, they are credited in Co-authored-by commit trailers.
  • Create a new changelog file in the changelog directory, with a name like <ISSUE NUMBER>.<TYPE>.rst.
  • Add yourself to AUTHORS in alphabetical order.

@psf-chronographer psf-chronographer Bot added the bot:chronographer:provided (automation) changelog entry is part of PR label Apr 27, 2026
@hamza-mobeen
Copy link
Copy Markdown
Contributor Author

@RonnyPfannschmidt I'm still waiting on your feedback :)

Copy link
Copy Markdown
Member

@RonnyPfannschmidt RonnyPfannschmidt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This. Technically looks good but is also at odds with a number of comments on the original issue

As such id like to defer to @nicoddemus

Copy link
Copy Markdown
Member

@nicoddemus nicoddemus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @hamza-mobeen for the PR, overall great work!

I left some comments, please take a look.

Comment thread doc/en/how-to/output.rst Outdated
Comment thread doc/en/reference/reference.rst Outdated
Comment thread doc/en/reference/reference.rst Outdated
Comment thread src/_pytest/assertion/__init__.py Outdated
Comment on lines +226 to +227
saved_config = util._config
util._config = config
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this necessary? Is the config being passed here different from the one already in util._config?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, given this is the only place where util.assertrepr_compare is called, consider just passing the text diff style directly to util.assertrepr_compare.

Comment thread src/_pytest/assertion/util.py Outdated
Comment thread src/_pytest/assertion/util.py Outdated
) -> list[str]:
if (
assertion_text_diff_style == ASSERTION_TEXT_DIFF_STYLE_BLOCK
and _is_multiline_text(left, right)
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure we should special case being multiline text or not; I think it is reasonable to always honor the block configuration, even for single line texts.

Comment thread src/_pytest/assertion/util.py Outdated
right: str,
highlighter: _HighlightFunc,
verbose: int,
assertion_text_diff_style: str,
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of typing this as str, let's use Literal["ndiff", "block"]. This way we are more explicit, plus we can use match below.

hamza-mobeen and others added 8 commits May 18, 2026 08:37
Co-authored-by: Bruno Oliveira <bruno@soliv.dev>
Co-authored-by: Bruno Oliveira <bruno@soliv.dev>
Co-authored-by: Bruno Oliveira <bruno@soliv.dev>
Co-authored-by: Bruno Oliveira <bruno@soliv.dev>
Co-authored-by: Codex <codex@openai.com>
Co-authored-by: Codex <codex@openai.com>
Copy link
Copy Markdown
Member

@nicoddemus nicoddemus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

Comment thread src/_pytest/assertion/util.py
Comment thread src/_pytest/assertion/util.py
nicoddemus and others added 3 commits May 18, 2026 12:52
Comment thread src/_pytest/assertion/util.py Outdated
@nicoddemus nicoddemus merged commit d6dbe52 into pytest-dev:main May 18, 2026
33 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bot:chronographer:provided (automation) changelog entry is part of PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

assertion diffs for multiline-string can become unreadable soup

3 participants